Semi-local variational optical flow estimation

International audienceGlobal variational methods for optical flow estimation usually suffer from an over-smoothing effect. We propose a semilocal estimation framework designed to integrate and improve any variational method. The idea is to implicitly segment the minimization domain into coherently moving windows. In a first time, local variational estimations are performed in overlapping candidate square regions. Then, a global discrete optimization, non subject to the over-smoothing introduced by variational approaches, selects the optimal window for each pixel. Experimental results show an increasing of the sharpness of discontinuities and a significant improvement of global registration errors compared to the results of the baseline global variational method

Spatially-variant kernel for optical flow under low signal-to-noise ratios: application to microscopy

International audienceLocal and global approaches can be identified as the two main classes of optical flow estimation methods. In this paper, we propose a framework to combine the advantages of these two principles, namely robustness to noise of the local approach and discontinuity preservation of the global approach. This is particularly crucial in biological imaging, where the noise produced by microscopes is one of the main issues for optical flow estimation. The idea is to adapt spatially the local support of the local parametric constraint in the combined local-global model [6]. To this end, we jointly estimate the motion field and the parameters of the spatial support. We apply our approach to the case of Gaussian filtering, and we derive efficient minimization schemes for usual data terms. The estimation of a spatially varying standard deviation map prevents from the smoothing of motion discontinuities, while ensuring robustness to noise. We validate our method for a standard model and demonstrate how a baseline approach with pixel-wise data term can be improved when integrated in our framework. The method is evaluated on the Middlebury benchmark with ground truth and on real fluorescence microscopy data

Aggregation of local parametric candidates with exemplar-based occlusion handling for optical flow

International audienceHandling all together large displacements, motion details and occlusions remains an open issue for reliable computation of optical flow in a video sequence. We propose a two-step aggregation paradigm to address this problem. The idea is to supply local motion candidates at every pixel in a first step, and then to combine them to determine the global optical flow field in a second step. We exploit local parametric estimations combined with patch correspondences and we experimentally demonstrate that they are sufficient to produce highly accurate motion candidates. The aggregation step is designed as the discrete optimization of a global regularized energy. The occlusion map is estimated jointly with the flow field throughout the two steps. We propose a generic exemplar-based approach for occlusion filling with motion vectors. We achieve state-of-the-art results in computer vision benchmarks, with particularly significant improvements in the case of large displacements and occlusions

A Variational Aggregation Framework for Patch-Based Optical Flow Estimation

International audienceWe propose a variational aggregation method for optical flow estimation. It consists of a two-step framework, first estimating a collection of parametric motion models to generate motion candidates, and then reconstructing a global dense motion field. The aggregation step is designed as a motion reconstruction problem from spatially varying sets of motion candidates given by parametric motion models. Our method is designed to capture large displacements in a variational framework without requiring any coarse-to-fine strategy. We handle occlusion with a motion inpainting approach in the candidates computation step. By performing parametric motion estimation, we combine the robustness to noise of local parametric methods with the accuracy yielded by global regularization. We demonstrate the performance of our aggregation approach by comparing it to standard variational methods and a discrete aggregation approach on the Middlebury and MPI Sintel datasets

Fast and robust single particle reconstruction in 3D fluorescence microscopy

Single particle reconstruction has recently emerged in 3D fluorescence microscopy as a powerful technique to improve the axial resolution and the degree of fluorescent labeling. It is based on the reconstruction of an average volume of a biological particle from the acquisition multiple views with unknown poses. Current methods are limited either by template bias, restriction to 2D data, high computational cost or a lack of robustness to low fluorescent labeling. In this work, we propose a single particle reconstruction method dedicated to convolutional models in 3D fluorescence microscopy that overcome these issues. We address the joint reconstruction and estimation of the poses of the particles, which translates into a challenging non-convex optimization problem. Our approach is based on a multilevel reformulation of this problem, and the development of efficient optimization techniques at each level. We demonstrate on synthetic data that our method outperforms the standard approaches in terms of resolution and reconstruction error, while achieving a low computational cost. We also perform successful reconstruction on real datasets of centrioles to show the potential of our method in concrete applications

Imaging neural activity in the ventral nerve cord of behaving adult Drosophila

To understand neural circuits that control limbs, one must measure their activity during behavior. Until now this goal has been challenging, because limb premotor and motor circuits have been largely inaccessible for large-scale recordings in intact, moving animals-a constraint that is true for both vertebrate and invertebrate models. Here, we introduce a method for 2-photon functional imaging from the ventral nerve cord (VNC) of behaving adult Drosophila melanogaster. We use this method to reveal patterns of activity across nerve cord populations during grooming and walking and to uncover the functional encoding of moonwalker ascending neurons (MANs), moonwalker descending neurons (MDNs), and a previously uncharacterized class of locomotion-associated A1 descending neurons. Finally, we develop a genetic reagent to destroy the indirect flight muscles and to facilitate experimental access to the VNC. Taken together, these approaches enable the direct investigation of circuits associated with complex limb movements

Méthode d'Agrégation et Représentation d'Images par Motifs pour le Flot Optique

This thesis is concerned with dense motion estimation in image sequences, also known as optical flow. Usual approaches exploit either local parametrization or global regularization of the motion field. We explore several ways to combine these two strategies, to overcome their respective limitations. We first address the problem in a global variational framework, and consider local filtering of the data term. We design a spatially adaptive filtering optimized jointly with motion, to prevent over-smoothing induced by the spatially constant approach. In a second part, we propose a generic two-step aggregation framework for optical flow estimation. The most general form is a local computation of motion candidates, combined in the aggregation step through a global model. Large displacements and motion discontinuities are efficiently recovered with this scheme. We also develop a generic exemplar-based occlusion handling to deal with large displacements. Our method is validated with extensive experiments in computer vision benchmarks. We demonstrate the superiority of our method over state-of-the-art on sequences with large displacements. Finally, we adapt the previous methods to biological imaging issues. Estimation and compensation of large local intensity changes frequently occurring in fluorescence imaging are efficiently estimated and compensated with an adaptation of our aggregation framework. We also propose a variational method with local filtering dedicated to the case of diffusive motion of particles.Nous nous intéressons dans cette thèse au problème de l’estimation dense du mouvement dans des séquences d’images, également désigné sous le terme de flot optique. Les approches usuelles exploitent une paramétrisation locale ou une régularisation globale du champ de déplacement. Nous explorons plusieurs façons de combiner ces deux stratégies, pour surmonter leurs limitations respectives.Nous nous plaçons dans un premier temps, dans un cadre variationnel global, et considérons un filtrage local du terme de données. Nous proposons un filtrage spatialement adaptatif, optimisé conjointement au mouvement, pour empêcher le sur-lissage induit par le filtrage spatialement constant.Dans une seconde partie, nous proposons un cadre générique d’agrégation pour lestimation du flot optique. Sous sa forme générale, il consiste en une estimation locale de candidats de mouvements, suivie de leur combinaison à l’étape d’agrégation avec un modèle global. Ce schéma permet une estimation efficace des grands déplacements et des discontinuités de mouvement. Nous développons également une méthode générique de gestion des occultations. Notre méthode est validée par une analyse expérimentale poussée sur les bases de données de référence en vision par ordinateur. Nous démontrons la supériorité de notre méthode par rapport à l’état de l’art sur les séquences contenant de grands déplacements.La dernière partie de la thèse est consacrée à l’adaptation des approches précédentes à des problématiques d’imagerie biologique. Les changements locaux importants d’intensité produits en imagerie de fluorescence sont estimés et compensé par une adaptation de notre schéma d’agrégation. Nous proposons également une méthode variationnelle avec filtrage local dédiée au cas de mouvements diffusifs de particules

Méthode d'Agrégation et Représentation d'Images par Motifs pour le Flot Optique

This thesis is concerned with dense motion estimation in image sequences, also known as optical flow. Usual approaches exploit either local parametrization or global regularization of the motion field. We explore several ways to combine these two strategies, to overcome their respective limitations. We first address the problem in a global variational framework, and consider local filtering of the data term. We design a spatially adaptive filtering optimized jointly with motion, to prevent over-smoothing induced by the spatially constant approach. In a second part, we propose a generic two-step aggregation framework for optical flow estimation. The most general form is a local computation of motion candidates, combined in the aggregation step through a global model. Large displacements and motion discontinuities are efficiently recovered with this scheme. We also develop a generic exemplar-based occlusion handling to deal with large displacements. Our method is validated with extensive experiments in computer vision benchmarks. We demonstrate the superiority of our method over state-of-the-art on sequences with large displacements. Finally, we adapt the previous methods to biological imaging issues. Estimation and compensation of large local intensity changes frequently occurring in fluorescence imaging are efficiently estimated and compensated with an adaptation of our aggregation framework. We also propose a variational method with local filtering dedicated to the case of diffusive motion of particles.Nous nous intéressons dans cette thèse au problème de l’estimation dense du mouvement dans des séquences d’images, également désigné sous le terme de flot optique. Les approches usuelles exploitent une paramétrisation locale ou une régularisation globale du champ de déplacement. Nous explorons plusieurs façons de combiner ces deux stratégies, pour surmonter leurs limitations respectives.Nous nous plaçons dans un premier temps, dans un cadre variationnel global, et considérons un filtrage local du terme de données. Nous proposons un filtrage spatialement adaptatif, optimisé conjointement au mouvement, pour empêcher le sur-lissage induit par le filtrage spatialement constant.Dans une seconde partie, nous proposons un cadre générique d’agrégation pour lestimation du flot optique. Sous sa forme générale, il consiste en une estimation locale de candidats de mouvements, suivie de leur combinaison à l’étape d’agrégation avec un modèle global. Ce schéma permet une estimation efficace des grands déplacements et des discontinuités de mouvement. Nous développons également une méthode générique de gestion des occultations. Notre méthode est validée par une analyse expérimentale poussée sur les bases de données de référence en vision par ordinateur. Nous démontrons la supériorité de notre méthode par rapport à l’état de l’art sur les séquences contenant de grands déplacements.La dernière partie de la thèse est consacrée à l’adaptation des approches précédentes à des problématiques d’imagerie biologique. Les changements locaux importants d’intensité produits en imagerie de fluorescence sont estimés et compensé par une adaptation de notre schéma d’agrégation. Nous proposons également une méthode variationnelle avec filtrage local dédiée au cas de mouvements diffusifs de particules